Motion coordination operation device and method, program, and motion coordination reproduction system

ABSTRACT

A motion coordination operation device is provided, which includes a movement information input unit inputting movement information supplied from respective sensor devices mounted on a plurality of users; and an operation unit operating coordination information that indicates the synchronization of motions of bodies or parts of the bodies of the plurality of users from the input movement information from the respective sensor devices, and generating control information for feedback outputs for the users based on the coordination information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 12/905,332,filed Oct. 15, 2010, which claims the benefit of priority under 35U.S.C. §119 from Japanese Patent Application No. 2009-244304, filed Oct.23, 2009, the entire contents of both of which are incorporated hereinby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a motion coordination operation deviceand method and a program, which operate coordination information thatindicates the consistency of movements of plural users. Also, thepresent invention relates to a motion coordination reproduction systemwhich performs reproduction based on motion coordination informationwith respect to respective users using the motion coordination operationdevice.

2. Description of the Related Art

In Japanese Registered Utility Model No. 3083641, a device which canmake even one person learn dance steps by detecting the consistency ofdance rhythms by a pressure sensor and displaying the same by LEDs isdisclosed as a dance step exercise machine.

In Japanese Unexamined Patent Application Publication No. 2001-293246,Japanese Registered Utility Model No. 3098422, Japanese UnexaminedPatent Application Publication No. 2007-293042, and Japanese UnexaminedPatent Application Publication No. 2007-236765, an appliance thatperforms evaluation with respect to a user's input timing in a rhythmgame, a music game, and the like, a device that plays on an artificialmusical instrument using an acceleration sensor as an input device, andthe like, are disclosed.

In Japanese Patent No. 3685731, a device that considers the correlationbetween respective manipulation information for plural users as thetarget of the determination is disclosed. In this case, the timing ofthe switch manipulation of the plural users is evaluated with respect tothe performance of the musical accompaniment.

SUMMARY OF THE INVENTION

For example, in the case of harmonized movements such as a dance byplural users, it is difficult for a person who is performing themovements to evaluate whether the movements coincide with one another.Up till now, a third party checks the movements, or the movements arefilmed on video to be evaluated later.

Here, it is effective in practicing dances and the like to evaluate thesynchronization of movements of the plural user bodies or to feed thesynchronization back to the users.

Dances or the like put emphasis on how the movements of plural personsare made to coincide with one another, and a system, which simplyevaluates the harmony of the plural persons and receives the feedbackthereof, makes it possible to quickly learn the harmonized motion.

Also, by applying such a system to various other kinds of sports, games,and amusements, it is considered that the system is appropriate tofunctional proficiency or contributes to increase of enjoyment.

In the related art, on the point of evaluation of a user's motion ormovement, there are techniques used in game machines and the like as inthe above-described patent documents. However, according to thetechniques described in Japanese Registered Utility Model No. 3083641,Japanese Unexamined Patent Application Publication No. 2001-293246,Japanese Registered Utility Model No. 3098422, Japanese UnexaminedPatent Application Publication No. 2007-293042, and Japanese UnexaminedPatent Application Publication No. 2007-236765, the synchronization ofthe motions of the bodies of the plural users, for example, motions suchas dances and the like, is not evaluated.

Also, according to Japanese Patent No. 3685731, the timing of the switchmanipulation of the plural users is evaluated. However, this is toevaluate the manipulation timing to the end, but not to evaluate thesynchronization of the motions of the users' bodies.

That is, in the case of dances by plural users, a system which evaluateswhether each person dances in synchronization with another person andmakes each person recognize the result of the evaluation does not exist.

In view of the above situation, it is desirable to make it possible toprovide a device and a system which can realize the functionalproficiency or increase enjoyment by determining the synchronization ofthe motions of plural users, for example, the rhythms, patterns, and thelike, of the body movements, as feedback to the users.

According to an embodiment of the present invention, there is provided amotion coordination operation device, which includes a movementinformation input unit inputting movement information supplied fromrespective sensor devices mounted on a plurality of users; and anoperation unit operating coordination information that indicates thesynchronization of motions of the bodies or parts of the bodies of theplurality of users from the input movement information from therespective sensor devices, and generating control information forfeedback outputs for the users based on the coordination information.

The motion coordination operation device according to an embodiment ofthe present invention further includes a recording unit; wherein therecording unit records the movement information input from therespective sensor devices to the movement information input unit, andthe operation unit reads the movement information from the recordingunit at predetermined intervals and performs the operation of thecoordination information.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit calculates individualinformation (for example, a movement rhythm, a movement tempo, amovement pattern, a rhythm pattern of the movement, or a movementstrength), as the movements of the respective users, from the movementinformation provided from the respective sensor devices, and calculatesthe coordination information by an operation using the individualinformation.

Also, in the motion coordination operation device according to anembodiment of the present invention, the operation unit determines basicmovement information, as a group of all the users, from the individualinformation of the respective users, and calculates the coordinationinformation by an operation using the basic movement information and theindividual information.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit calculates the coordinationinformation, from the whole group of all the users, by the operationusing the individual information of the respective users.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit calculates the coordinationinformation by comparing the individual information of the respectiveusers with individual information of a specified user.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit calculates the coordinationinformation by comparing preset guidance movement information withindividual information of the respective users.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit generates the controlinformation for performing different feedback outputs for the respectiveusers.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit generates the controlinformation for performing the same feedback output for the respectiveusers.

In the motion coordination operation device according to an embodimentof the present invention, the operation unit generates drive controlinformation of a device that gives a tactile impetus to the users ascontrol information for the feedback output for the users.

According to another embodiment of the present invention, there isprovided a motion coordination operation method in an operationprocessing device capable of performing information input/output andoperation processes, which includes the steps of inputting movementinformation supplied from respective sensor devices mounted on aplurality of users; operating coordination information that indicatesthe synchronization of motions of bodies or parts of the bodies of theplurality of users from the input movement information from therespective sensor devices; and generating control information forfeedback outputs for the users based on the coordination information.

According to still another embodiment of the present invention, there isprovided a program that makes an operation processing device execute themotion coordination operation method.

According to still another embodiment of the present invention, there isprovided a motion coordination reproduction system, which includes aplurality of sensor devices, the motion coordination operation device,and a feedback device. Each of the sensor devices includes a detectionunit detecting user movement information, and an output unit outputtingthe movement information detected by the detection unit. The feedbackdevice includes a reproduction control unit controlling the driving of areproduction device for the users based on the control information.

That is, according to the embodiments of the present invention, themotions of the plural users are detected as the movement information,and the coordination information that indicates the synchronization ofthe motions of the plural users is obtained from the movementinformation detected for the respective users. The coordinationinformation means information that indicates whether the motions of therespective users (for example, body motions such as dances and the like)coincide with each other are a correlation or a discrepancy.

Also, feedback is given to the respective users based on thecoordination information. For example, it is possible to inform theusers of “the coincidence” of the motions, “the non-coincidence” of themotions, or the degree of deviation of the motions, by an impetus(vibration or the like) that the user can recognize through a tactilesensation.

According to the embodiments of the present invention, by givingfeedback to the users on the basis of the coordination information basedon the rhythms, patterns, strengths, and the like, of the movements ofthe plural users, the user knows the correlation (synchronization) toother users while performing the motion. Accordingly, it becomespossible to quickly learn the harmony of the motions or to increaseenjoyment as an amusement.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a motion coordination reproduction systemaccording to an embodiment of the present invention;

FIG. 2 is an explanatory diagram illustrating an example of a motioncoordination operation according to an embodiment of the presentinvention;

FIG. 3 is an explanatory diagram illustrating an example of a motioncoordination operation according to an embodiment of the presentinvention;

FIG. 4 is an explanatory diagram illustrating an example of a motioncoordination operation according to an embodiment of the presentinvention;

FIG. 5 is a flowchart illustrating a first processing example accordingto an embodiment of the present invention;

FIG. 6 is an explanatory diagram of a setting of a basic movement rhythmaccording to an embodiment of the present invention;

FIG. 7 is an explanatory diagram of the consistency/inconsistency ofmovement tempos according to an embodiment of the present invention;

FIGS. 8A and 8B are explanatory diagrams of motion patterns and rhythmpatterns according to an embodiment of the present invention;

FIG. 9 is a flowchart illustrating a second processing example accordingto an embodiment of the present invention;

FIG. 10 is a flowchart illustrating a third processing example accordingto an embodiment of the present invention; and

FIG. 11 is a flowchart illustrating a fourth processing exampleaccording to an embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

1. System configuration

2. First processing example

3. Second processing example

4. Third processing example

5. Fourth processing example

6. Applications, modifications, and program

1. System Configuration

FIG. 1 is a block diagram of a motion coordination reproduction systemaccording to an embodiment of the present invention.

This motion coordination reproduction system includes a motioncoordination operation device 1, a plurality of sensor devices 2, afeedback device 3, one or a plurality of reproduction devices 4, and aguidance device 5.

Each sensor device 2 includes a detection unit 21 and a movementinformation output unit 22. The detection unit 21, for example, may bean acceleration sensor, a pressure sensor, an angular velocity sensor, avibration sensor, or the like, and detects a user's motion, accelerationaccording to the movement, or the like. The movement information outputunit 22 outputs signals detected by the detection unit 21 to the motioncoordination operation device 1 in real time.

Each of the plurality of sensor devices 2 may be a device which ismounted on the body of a user to be carried by the user or held by theuser in his/her hand. Further, it is also considered that the sensordevice is in the form of a seat or a support on which the user takesexercise.

In the case of carrying the sensor device on the body of the user, forexample, it is considered that the sensor device is in a form that canbe mounted in a necessary position, such as a wrist, an ankle, a neck,an arm, a trunk, a leg portion, and the like, by a mounting belt, atape, and the like. Also, the sensor device may be of a headphone typeor a cap type, which is mounted on a head portion. Further, the sensordevice may be of a necklet type that is put on a neck, a type that ismounted on clothes with a clip, a type that is put in a pocket, and thelike.

In any type, it is sufficient if the sensor device 2 can detect movementinformation in response to the movement of the whole body or a part ofthe body (for example, hand, foot, or the like) of the user.

It is also considered that the sensor device is not a dedicated sensordevice, but is a device carried by the user, for example, in the form ofa sensor device 2 built in a remote controller.

As an example, a plurality of sensor devices 2, rather than a singlesensor device 2, may be mounted on the right hand, left hand, rightfoot, and left foot of the user, respectively. That is, each user maycarry a plurality of sensor devices 2.

The movement information output unit 22 outputs signals according to theacceleration or pressure detected by the detection unit 21 as movementinformation, and performs communication with the motion coordinationoperation device 1 by either a wired output or a wireless output (radiowave signal, infrared signal, or the like).

Further, it is not necessary for the plurality of users to be in thesame place, and for example, the users may be in remote places,respectively. In this case, the respective sensor devices 2 may transmitthe movement information to the motion coordination operation device 1by network communication.

The movement information, which is data transmitted from the movementinformation output unit 22 to the motion coordination operation device1, may be data of detected values of the detection unit 21 or encodeddata.

In this case, in order for the motion coordination operation device 1 toidentify the movement information of the respective users, it isappropriate to include an ID for the sensor device 2 or a user ID in thetransmitted data.

However, in the case of a wired transmission or even if the ID is notused through the setting of a modulation frequency, it is possible thatthe motion coordination operation device 1 identifies which sensordevice 2 the movement information is transmitted from.

The motion coordination operation device 1 includes a motion informationinput unit 14, a recording unit 15, an operation unit 11, and a controlinformation output unit 13. According to the motion coordinationoperation device 1, the motion information input unit 14, the recordingunit 15, the operation unit 11, and the control information output unit13 are realized on a hardware configuration such as a CPU, a RAM (RandomAccess Memory), a ROM (Read Only Memory), an input/output interface, andthe like. Accordingly, the motion coordination operation device 1 may beconfigured as a general operation processing device such as a personalcomputer or the like. Of course, the motion coordination operationdevice 1 may be configured as a dedicated device.

The movement information input unit 14 receives the movement informationfrom the respective sensor devices 2 through wired, wireless, or networkcommunications. Also, the movement information input unit 14 performsdemodulation or the like according to the communication system.

The movement information received from the respective sensor devices 2into the movement information input unit 14 is recorded in the recordingunit 12. The recording unit 12 may include a RAM, an HDD (Hard DiskDrive), other kinds of recording media, and their recording/reproductionprocessing systems.

The recording unit 12 records the received movement information, andsimultaneously reproduces the movement information from the respectivesensor devices 2 in a predetermined period according to the request fromthe operation unit 11 to transmit the movement information to theoperation unit 11.

The operation unit 11 performs an operation process using the movementinformation as in the first to fourth processing examples to bedescribed later. That is, the operation unit 11 operates coordinationinformation that indicates the synchronization of the bodies or parts ofthe bodies of the plural users from the movement information from therespective sensor devices 2, and generates control information forfeedback output for the users based on the coordination information.

For example, in the case where the movement information is informationthat indicates the motion of the users detected by the accelerationsensor, the operation unit 11 is able to obtain peak-to-peak time of theacceleration or know the strength of the movement, rather than therhythm, tempo, timing or amplitude value of the movement (in the case ofwalking/running, landing of the heel) from the peak of a waveform of theacceleration, by performing spectrum analysis or operation of selfcorrelation using the acceleration.

Also, the movement pattern may be known from the pattern of the existingwaveform of the acceleration.

The operation unit 11 may detect whether the motions of the plural userscoincide with one another or whether the motions exist in a preset rangeof several hundred milliseconds to several seconds on the basis of themovement tempo detected by the acceleration sensor.

The synchronization of the motions of the users is calculated as thecoordination information.

In the detailed examples to be described later, the expressions“correlation” and “discrepancy” are used as the coordinationinformation. Both the correlation and the discrepancy are indexes thatindicate the synchronization of the plural users, and their relation isdiscrepancy=1/correlation.

Specifically, for example, the movement tempo, the movement strength,the movement region, and the movement pattern are estimated, for aperiod of several hundred milliseconds to several seconds, from themovement information of the acceleration sensor, and whether there is arelation between the plurality of users for a period of several framesin a time series is determined. The correlation and the discrepancy areexpressed as functions of the movement tempo, the movement strength, themovement range, the movement pattern, the number of coordinationpersons, and the number of uncoordinated persons.

Further, the operation unit 11 generates a control signal for feedbackreproduction that makes the user recognize the situation of thesynchronization based on the coordination information.

In the description of the present invention, the feedback reproductionmeans the reproduction that presents the synchronization or the degreeof synchronization between the user motions to the users by tactileimpetus, video, sound, or the like.

The control information for the feedback reproduction generated by theoperation unit 11 is supplied from the control information output unit13 to the feedback device 3.

The feedback device 3 is configured integrally with the motioncoordination operation device 1 or separately from the motioncoordination operation device. In particular, the feedback device 3 is adevice that drives the reproduction device 4, and due to its necessity,it may be a device separate from the motion coordination operationdevice 1. That is, the feedback device 3 may be arranged in a positionthat is set apart from the motion coordination operation device 1, inthe sense that the feedback device 3 is installed in a position wherethe user exists.

The feedback device 3 includes a control information input unit 3 a anda feedback reproduction control unit 3 b.

The control information input unit 3 a receives and inputs the controlinformation from the control information output unit 13 of the motioncoordination operation unit 1.

The communication between the control information output unit 13 and thecontrol information input unit 3 a may be any one of a wiredcommunication, a wireless communication, and a network communication. Ifthe motion coordination operation device 1 and the feedback device 3 arecombined into an appliance, the communication may be realized as thetransmission of internal control information of the correspondingappliance.

The feedback reproduction control unit 3 controls the driving of thereproduction device 4 based on the control information received by thecontrol information input unit 3 a.

As an example, the reproduction device 4 may be a device that gives atactile impetus to the user. For example, it is assumed that thereproduction device 4 is a device that is mounted on the user's body,arm, wrist, leg portion, or the like, to give the rhythm, vibration,pressure, or the like, to the user. In the case of such a reproductiondevice 4, it may be considered that the reproduction device 4 and thesensor device 2 are integrally configured into a device.

Also, the reproduction device 4 may be a video monitor device, an audiospeaker device, a headphone device, or the like, and present informationon the users by video or audio.

That is, the reproduction device 4 may be a so-called mount type deviceor a device carried on the user's body.

Also, the reproduction device 4 may perform the reproduction, which maybe any one of a tactile impetus reproduction, an audio reproduction, anda video reproduction, with respect to each user or with respect to allthe users.

The guidance device 5 is a device that performs the guide output of theoperations of the users. The guidance device 5, for example, includes aguidance control unit 5 a and a guidance reproduction unit 5 b.

The guidance control unit 5 a controls the execution of the guidancereproduction. The guidance reproduction performs informationpresentation to the user as the guidance.

The guidance device 5 is not necessarily installed in terms of thesystem.

For example, in the case of a group dance, the guidance device 5 may beconsidered to reproduce the tempo of the dance, music, video, and thelike, through an audio speaker device or a monitor device as theguidance reproduction unit 5 b. In this case, the guidance device 5 isnot considered as a system configuration appliance, but is considered asan audio device or a video device of the group.

On the other hand, it is also considered to use the guidance device 5 inassociation with the system operation.

For example, in the reproduction device 4, it is also considered toperform the guidance reproduction that overlaps the feedbackreproduction.

In this case, the guidance control unit 5 a may perform a predeterminedcontrol by receiving the control information from the motioncoordination operation device 1, or request the feedback reproductioncontrol unit 3 b to perform the reproduction operation through thereproduction device 4 in addition to the guidance reproduction.

In such a motion coordination reproduction system, even if therespective devices have diverse shapes, arrangements, or are formedintegrally/separately, the synchronization of movements of all thebodies or parts of the bodies of the plural users is obtained as thecoordination information, and the feedback reproduction for the usersare performed appropriately. In order to realize the correspondingoperation, the device configuration and the communication type betweenthe devices may be appropriately changed according to the place wherethe respective users are.

Also, according to the purpose of the system operation, such as whatmovement the system is used for, which synchronization is to bedetermined, and the like, the configuration (the contents of detection)of the sensor device 2, the feedback reproduction type, and the like,are appropriately set.

Also, diverse operation process types of the motion coordinationoperation device 1 may be considered. Hereinafter, examples of theoperation process types are shown in FIGS. 2, 3, and 4.

FIG. 2 shows a case where each of users (users A, B, and C) has onesensor device 2 mounted thereon. In this case, an operation unit 11 ofthe motion coordination operation device 1 performs operations M1, M2,and M3.

As the operation M1, the operation unit 11 performs individualinformation operation for the users A, B, and C using the movementinformation from the respective sensor devices 2. The individualinformation is information obtained by estimating the personal motioninformation of the respective users. For example, the operation unit 11determines the movement rhythm, the movement tempo, the movementstrength, the movement pattern, and the movement rhythm pattern for eachperson.

Next, the operation unit 11 performs the coordination informationoperation as the operation M2. That is, the operation unit 11 calculatesthe correlation or the discrepancy from the respective user motions,i.e. motion information obtained by the individual informationoperation, as the synchronization between the respective users.

Last, the operation unit 11 performs the feedback control operation asthe operation M3. In the operation M3, the operation unit 11, forexample, generates the control information for the user tactile impetusreproduction according to the coordination information obtained from theoperation M2. In this case, the individual control information for therespective users may be generated or control information for the wholeuser group may be generated.

For example, as the individual control information, the controlinformation for expressing the deviation state of the respective usermovement rhythms as the feedback reproduction is generated for each userwith respect to the basic movement rhythm.

Also, as the control information for the whole group, the controlinformation for expressing the consistent state or the deviation stateas the feedback reproduction is generated as the synchronization of themovements of all the persons.

FIG. 3 shows the case where each user has a plurality of sensor devices2 mounted thereon. As an example, a user carries the sensor devices 2 onhis/her right hand, left hand, right foot, and left foot, respectively.Accordingly, the respective sensor devices 2 output movement informationas the movement of the right hand, the movement of the left hand, themovement of the right foot, and the movement of the left foot,respectively.

Even in this case, the operation unit 11 performs the individualinformation operations from the movement information from the respectivesensor devices 2 as the operation MIA. However, the individualinformation operations in this case become operations for obtainingmotion information of the respective portions of the body from themovement information from the respective sensor devices 2 mounted on therespective users.

That is, the operation unit 11 determines the movement rhythms, movementtempos, movement strengths, movement patterns of the right hand, lefthand, right foot, and left foot of the user A. The operation unit 11also determines the same with respect to the user B.

Then, the operation unit 11, as the operation M1B, performs the userinformation operation as the individual information in the unit of auser. For example, the operation unit 11 calculates the whole movementinformation of the user from the movement rhythms of the right hand,left hand, right foot, and left foot of the user A. For example, theoperation unit 11 obtains the rhythms or patterns of the whole movement.

Then, the operation unit 11 performs the coordination informationoperation as the operation M2. That is, the operation unit 11 calculatesthe correlation or the discrepancy from the respective user motions,i.e. respective motion information obtained by the individualinformation operation MIA and user information operation M1B, as thesynchronization between the respective users.

Last, the operation unit 11 performs the feedback control operation asthe operation M3. In the operation M3, the operation unit 11, forexample, generates the control information for the user tactile impetusreproduction according to the coordination information obtained from theoperation M2. Even in this case, the individual control information forthe respective users may be generated or control information for thewhole user group may be generated.

As shown in FIG. 3, it may be considered that the motions of a pluralityof portions of one user are detected as the movement information, thecomposite movement of the user body of one person is estimated from thedetected movement information, and then the coordination informationoperation is performed by comparing the composite movement of the bodybetween the plurality of users.

In the same manner, the operation process as shown in FIG. 4 isconsidered as an example of the user's mounting of the plurality ofsensor devices 2.

As in the case of FIG. 3, each user has a plurality of sensor devices 2mounted thereon.

The operation unit 11 performs the individual information operation fromthe movement information from the sensor devices 2 as the operation MIA.That is, the operation unit 11 performs the operation to obtain movementrhythms and so on, as the motion information of the respective portionsof the body, from the movement information from the respective sensordevices 2 mounted on the respective users.

Then, the operation unit 11 performs the coordination informationoperation for the respective portions of the user's body as theoperation M2A.

That is, the operation unit 11 compares the movement rhythms of theright hands of the respective users. Also, the operation unit 11compares the movement rhythms of the right foots of the respectiveusers. Also, the operation unit 11 compares the movement rhythms of theleft foots of the respective users. Also, the operation unit 11 comparesthe movement rhythms of the left hands of the respective users.

As described above, the operation unit 11 calculates the correlation orthe discrepancy of the motions for each body region.

Then, the operation unit 11 performs the composite coordinationinformation operation of the users as the operation M2B. That is, theoperation unit 11 calculates the composite correlation or discrepancyusing the correlation or discrepancy values of the right hand, the rightfoot, the left foot, and the left hand of the respective users.

Last, the operation unit 11 performs the feedback control operation asthe operation M3. In the operation M3, the operation unit 11, forexample, generates the control information for the user tactile impetusreproduction according to the composite coordination informationobtained from the operation M2B. Even in this case, the individualcontrol information for the respective users may be generated or thecontrol information for the whole user group may be generated.

As shown in FIG. 4, it may be considered that the motions of a pluralityof portions of one user are detected as the movement information, andfrom them, coordination information is operated for the body portions ofthe respective users, and then the coordination information operationsare performed between the plurality of users, from the coordinationinformation of the respective portions of the bodies. In this case, thecorrelation or discrepancy of the respective portions of the body isweighted, and then the composite coordination information operation isperformed.

The above-described examples are merely examples of the operationtechniques, and other diverse examples may be considered. In accordancewith the kind of the sensor device, the number of sensor devices beingused, the contents of detection, mounting position, the contents ofmovement, the purpose of the feedback reproduction, and the like, theoperation technique may be selected.

2. First Processing Example

Hereinafter, the detailed processing examples of the motion coordinationoperation device 1 will be described. It is assumed that the motioncoordination reproduction system performs the feedback reproduction withrespect to the respective users in the case where a plurality of userspractices gymnastics or dances as a group.

FIG. 5 shows the processing of the motion coordination operation device1.

It is assumed that the plurality of users dances, for example, with therespective sensor devices 2 mounted on or carried by the users, as shownin FIG. 2. The sensor device 2, for example, includes a detection unit21 as the acceleration sensor, and outputs in real time the detectedacceleration information through the movement information output unit 22as the movement information.

In step F101 of FIG. 5, the motion coordination operation device 1starts the recording of the movement information. That is, if the motioninformation input unit 14 of the motion coordination operation device 1receives the movement information from the respective sensor devices 2,the motion coordination operation device 1 records the received motioninformation in the recording unit 12. Thereafter, such a recordingprocess is continuously performed.

In this case, the movement information from the respective sensordevices 2 is recorded so that it is discriminated which sensor device(which user) the movement information is from. For example, as describedabove, the recording is performed to make the IDs (identificationinformation) of the sensor devices 2 correspond to the respectivemovement information.

In step F102, the operation unit 11 reads the movement information for apredetermined period with respect to the movement information from therespective sensor devices 2, which is recorded in the recording unit 12.The predetermined period, for example, is a set period such as 10seconds, 30 seconds, one minute, or the like.

Although periods t1, t2, t3 and so on are illustrated in FIG. 7, theoperation unit 11 reads the movement information from the recording unit12 in the priority period t1.

In step F103, the operation unit 11 performs the individual informationoperation of the users of the respective sensor devices 2 using themovement information from the respective sensor devices 2, which is readfrom the recording unit 12. Here, it is exemplified that the estimationof the movement rhythms is performed.

If the movement performed by the users is a periodic movement, a tempo(for example, the number of beats per minute (BPM)) may be used as theestimation of the movement rhythm.

For example, as the individual information operation, the tempo (BPM) ofeach user in an object period (t1: for example, one minute) this time isestimated from the movement information of the acceleration sensor orthe like. Specifically, the tempo is calculated by detecting peak valuesof the acceleration sensor for one minute. The peak detection ispossible by performing spectrum analysis of the acceleration values orself-correlation operation thereof.

The above-described calculation is performed with respect to therespective users (movement information from the respective sensordevices 2).

In this case, it may be considered to calculate the movement pattern,the rhythm pattern, or the movement strength, rather than the movementrhythm (tempo), in accordance with the user's motion, the kind ofgymnastics or dances, and other movement types.

For example, FIG. 8A shows acceleration sensor values for the users Aand B. On each sensor device 2 mounted on the respective users, anacceleration sensor in X, Y, and Z directions is mounted, and waveformsindicated by a solid line, a dashed line, and a dashed dot line in thedrawing are obtained as the acceleration sensor values (movementinformation) in the respective direction. The movement pattern may beknown by the existing waveform pattern of the acceleration sensorvalues.

Also, FIG. 8B shows the rhythm of user A. For example, by detectingpeaks of the acceleration sensor values from the sensor device 2 mountedon the user A, the rhythm pattern of the user A may be detected. In thiscase, the rhythm pattern is not a simple tempo as the rhythm, butindicates the time length change between the respective beat timings.

Further, although not illustrated, the movement strength or the strengthpattern may be detected from the amplitude of the waveform of theacceleration sensor.

In step F103 of FIG. 5, the tempo (movement rhythm) and so on iscalculated as the individual information of each user.

Then, in step F104, the operation unit 11 estimates the basic movementrhythm in a group as the plural users. For example, in the case ofestimating the tempo of each user as the movement rhythm, a tempo thatis the comparison basis of the tempo is determined as the basic movementrhythm.

This basic movement rhythm is a representative movement rhythm in thegroup. For example, with respect to the movement rhythm of theindividual user calculated in step F103, the distribution as shown inFIG. 6 is discriminated. Since the movement rhythm (tempo (BPM))somewhat differs for each user, the distribution as shown in FIG. 6 isobtained, and the tempo having the highest frequency, that is, the tempohaving the largest number of users is determined as the basic movementrhythm.

In this case, regardless of the frequency (the number of users), it maybe considered that the basic movement rhythm is estimated as an averagevalue of tempos of all the users, or the basic movement rhythm iscalculated by correcting the tempo having the highest frequency to anaverage value and so on.

Any way, it is sufficient if the basic movement rhythm is to estimatethe tempo and so on, which all the users desire to make in coincidencewith the tempo.

In step F105 of FIG. 5, the basic movement rhythm and the discrepancyare calculated with respect to the movement rhythms of the respectiveusers. The discrepancy may be calculated as follows.

First, if the movement rhythm (tempo (BPM)) of a certain user is higherthan the basic movement rhythm (tempo (BPM), it is assumed that “tempoα”is the users movement rhythm, and “tempoβ” is the basic movement rhythm.

On the other hand, if the movement rhythm (tempo (BPM)) of the user islower than the basic movement rhythm (tempo (BPM), it is assumed that“tempoα” is the basic movement rhythm, and “tempoβ” is the movementrhythm of the user. That is, the faster one becomes “tempoα”.

Also, the discrepancy is obtained as follows.

Discrepancy=tempoα/tempoβ−int(tempoα/tempoβ)

Here, “int” indicates an integer value part.

For example, if it is assumed that the tempo of the movement rhythm of acertain user=60, and the tempo of the basic movement rhythm=56, thediscrepancy becomes 60/56-int(60/56). If 60/56=1.07, int(60/56) becomes“1” that is the integer part of 1.07.

Accordingly, the discrepancy is obtained as Discrepancy=1.07−1=0.07.

Also, for example, if it is assumed that the tempo of the movementrhythm of another user=48, and the tempo of the basic movementrhythm=56, the discrepancy becomes 56/48-int(56/48). If 56/48=1.16, thediscrepancy is obtained as 1.16−1=0.16.

Also, if the movement rhythm of another user is equal to the tempo (=56)of the basic movement rhythm, the discrepancy becomes 1−1=0.

As a result, the discrepancy value becomes larger as the discrepancybetween the user's tempo and the tempo of the basic movement rhythmbecomes greater.

The operation unit 11 obtains the discrepancy with respect to themovement rhythms of the respective users.

In step F106, the operation unit 11 generates the feedback controlinformation, and transmits the feedback control information from thecontrol information output unit 13 to the feedback device 3.

The feedback control information is generated according to thediscrepancy.

As an example, the feedback reproduction is to transfer the basicmovement rhythm to the user as the tactile impetus. For example, it isassumed that the reproduction device 4 may be a pressure device, avibrator, or the like, that is mounted on the skin of the user.

Also, it is assumed that the basic movement rhythm is transferred by thereproduction device 4. That is, the tactile impetus is given to the useras the tempo of the basic movement rhythm.

Here, for each user, the strength of the tactile impetus is setaccording to the value of the discrepancy.

That is, the operation unit 11, as the feedback control information,generates tempo information as the basic movement rhythm and thestrength coefficient according to the discrepancy value for each user,and outputs their control information to the feedback device 3.

The strength coefficient may be the discrepancy value itself or a valuethat is obtained by multiplying the discrepancy value by a predeterminedcoefficient, and further, may be a value obtained by multiplying them bya time value. For example, although it is expected that the discrepancyvalue is changed in the process of movement, the length of a period, inwhich the discrepancy value that exceeds the predetermined value isobtained, is considered to be the coefficient. That is, as the period inwhich deviation of the motions occurs is longer, the strengthcoefficient becomes higher.

By such feedback control information, the feedback reproduction controlunit 3 b of the feedback device 3 controls the driving of the tactileimpetus reproduction with respect to the respective reproduction devices4 mounted on the respective users.

That is, although the tempo as the basic movement rhythm is transferredto the tactile sensation with respect to all the users, the tactileimpetus becomes greater as the deviation of the motions becomes larger.That is, the user who has larger deviation of the motions feels thetempo of the basic movement rhythm stronger.

At this time, although it is reasonable to do so for this, it isnecessary that the control information for the respective userscorrespond to the reproduction devices 4 mounted on the respectiveusers. The feedback reproduction control unit 3 b controls the drivingof the reproduction device 4 by setting the reproduction strength of thereproduction device 4 mounted on the user in accordance with the controlinformation (strength coefficient according to the discrepancy) for therespective users.

If the tempos for the respective users coincide with each other, theuser feels small with respect to the rhythm felt with tactile sensationby the reproduction device 4, while if the tempos deviate from eachother, the user feels large with respect to the rhythm felt with tactilesensation.

The operation unit 11, after the processing in step F106, returns fromstep F107 to step F102 to continue the same process. That is, during thecontinuous system operation, the operation unit 11 repeats performing ofsteps F102 to F106.

If the system operation is ended, the operation unit 11 proceeds fromstep F107 to step F108, and ends the recording process of the movementinformation in the recording unit 12, the operation process of theoperation unit 11, and the generation/output process of the feedbackcontrol information to terminate a series of operations.

An example of the system operation realized by the process of FIG. 5will be described with reference to FIG. 7.

As described above, the operation unit 11 performs the processing ofFIG. 5 with respect to the movement information of the respective usersfor each period of t1, t2, t3, or the like.

In FIG. 7, the vertical axis represents movement tempo, and thehorizontal axis represents time. A dashed line indicates the movementrhythm (tempo) of a certain user A, and a solid line indicates the basicmovement rhythm (tempo) estimated in step F104.

A period in which the tempo of the user A almost coincides with thebasic movement rhythm is indicated as “consistency” in the drawing. Aperiod in which the tempo of the user A deviates from the basic movementrhythm is indicated as “inconsistency”.

Also, when the user's movement tempo is n times (for example, doublespeed consistency) the tempo of the basic movement rhythm, it isindicated as “double speed consistency”.

Here, the user A feels small with respect to the basic movement rhythmby the tactile impetus reproduction in a period in which the user'smovement tempo is “consistency” or “double speed consistency”. On theother hand, the user A feels the basic movement rhythm emphasizedaccording to the amount of deviation (or time length in which the user'smovement tempo further deviates from the basic movement rhythm) in aperiod of “inconsistency”.

If the basic movement rhythm felt with tactile sensation becomesstronger, the user feels that its motion deviates from the basicmovement rhythm, and performs dancing with attention to the coincidenceof the tempo with the basic movement rhythm. By doing this, the motionmay be corrected so that the tempo coincides with the whole movement.That is, it is sufficient if the motion is performed so that the tactileimpetus is not felt as far as possible. In other words, the tactileimpetus reproduction, due to the size of the impetus, becomes thefeedback reproduction such as learning the user having the movementdeviation a correct tempo with the reproduction strength according tothe deviation state.

Accordingly, the user can be aware of the relation with other users asperforming the operation, and quickly learns the harmony of theoperations. Also, the user can take pleasure in increasing thecorrelation.

At this time, in the above-described example, although it is exemplifiedthat the discrepancy is

Discrepancy=tempoα/tempoβ−int(tempoα/tempoβ),

a predetermined threshold value X is set, and if X is

|tempoα/tempoβ−int(tempoα/tempoβ)|<X,

it corresponds to the “consistency”. If X is

|tempoα/tempoβ−int(tempoα/tempoβ)|≧X,

it corresponds to the “inconsistency”. In this case, the correlation ofthe motion can be expressed by two values (consistency/inconsistency).

In this case, it may be considered that the feedback reproduction iscontrolled by two stages or by on/off operations according to thestrength of the tactile impetus reproduction that is “consistency” and“inconsistency.

Also, the reproduction for which a specified tactile impetus is given tothe “consistent” user may be considered.

Also, the reproduction may be treated by correlation(correlation=1/discrepancy) rather than the discrepancy.

The correlation, for example, may be calculated by

tempoβ/tempoα−int(tempoβ/tempoα). A feedback reproduction in which asthe correlation becomes higher, the predetermined tactile impetus isgiven stronger.

Further, in the case of the double speed consistency as illustrated inFIG. 7, it is considered that it is more difficult to perform the tempoconsistency such as twice, three times, and the like. Thus, in the caseof the double speed consistency, it may be treated to have a highercorrelation.

Also, the feedback reproduction is not performed as the tactile impetusreproduction with the strength individually different for each user, butmay be commonly reproduced to all the users.

For example, in steps F105 and F106, the strength of the discrepancy isset according to the width of distribution of movement rhythms of therespective users for the basic movement rhythm.

That is, the strength coefficient of the feedback control information isheightened to the extent that the motions of the respective users getscattered.

Accordingly, all the users can strongly feel the tactile impetusreproduction as the deviation from all the users becomes greater, andcan bodily sense the coincident state of the motions of all the userssimultaneously moving.

As described above, it is exemplified that the discrepancy and thecorrelation are calculated from the tempo values as the movementrhythms. However, as described above, it may be considered to use themovement pattern or the rhythm pattern.

In the case of obtaining the correlation from the movement pattern, forexample, the maximum value of the mutual correlation coefficients of thewaveforms in the X, Y, and Z directions of the acceleration sensor ismultiplied by 1/lag. The lag is a time discrepancy of the maximumcorrelation coefficients.

If it is assumed that the irregular variations having two discretevalues are x(t) and y(t), their mutual correlation coefficient C_(xy)(r)is expressed as in Equation (1).

$\begin{matrix}{{{Cxy}(r)} = {\frac{1}{N}{\sum\limits_{t - n}^{N - r - 1}\; {{x(t)}{y\left( {t + r} \right)}}}}} & (1)\end{matrix}$

Here, r denotes a parameter that indicates a parallel movement on thetime axis of y(t) that is called a lag.

When the correlation is heightened, the waveforms become similar to eachother, and the time discrepancy becomes smaller.

Also, in the case of using the rhythm pattern, the correlation (or thediscrepancy) can be calculated by comparing the user's rhythm as shownin FIG. 8B with the rhythm pattern as the basic rhythm.

For example, the correlation can be obtained by a technique ofaccumulating the amount of deviation of the respective bit timings (peaktimings of the movements) for a predetermined period.

Even in the case of the movement strength, the correlation can beobtained in the same manner.

3. Second Processing Example

The second processing example of the motion coordination operationdevice 1 will be described with reference to FIG. 9.

Even in the second to fourth processing examples to be described later,in the same manner as the first processing example, it is assumed that aplurality of users dances, for example, with the respective sensordevices 2 mounted on or carried by the users as shown in FIG. 2. Thesensor device 2, for example, includes a detection unit 21 as theacceleration sensor, and outputs in real time the detected accelerationinformation through the movement information output unit 22 as themovement information.

In step F201 of FIG. 9, the motion coordination operation device 1starts the recording of the movement information. That is, if the motioninformation input unit 14 of the motion coordination operation device 1receives the movement information from the respective sensor devices 2,the motion coordination operation device 1 records the received motioninformation in the recording unit 12. Thereafter, such a recordingprocess is continuously performed.

In step F202, the operation unit 11 reads the movement information for apredetermined period with respect to the movement information from therespective sensor devices 2, which is recorded in the recording unit 12.

Also, in step F203, the operation unit 11 performs the individualinformation operation of the users of the respective sensor devices 2using the movement information from the respective sensor devices 2,which is read from the recording unit 12. For example, the operationunit 11 determines the movement tempos as the movement rhythms of therespective users.

The above-described steps are the same as steps F101 to F103 accordingto the processing example 1.

In step F204, the operation unit 11 calculates the discrepancy betweenthe respective users.

For example, in an example of three users A, B, and C, the discrepanciesbetween users A and B, between users B and C, and between users C and Aare calculated.

For example, if it is assumed that with respect to the movement temposbetween two users, “tempoα” is the side having a fast tempo, and“tempoβ” is the side having a slow tempo, the discrepancy is obtained asfollows.

Discrepancy=tempoα/tempoβ−int(tempoα/tempoβ)

If the discrepancies of the respective user combinations are obtained,the operation unit 11 calculates the discrepancies of the whole group instep F205. For example, an average value of the discrepancies of therespective combinations or the width of distribution becomes thediscrepancy of the whole group.

In step F206, the operation unit 11 generates the feedback controlinformation, and transmits the feedback control information from thecontrol information output unit 13 to the feedback device 3.

The feedback control information is generated according to thediscrepancy of the whole group.

For example, the operation unit 11 generates the feedback controlinformation having high reproduction strength coefficient as thediscrepancy of the whole group becomes larger (or smaller), and outputsthe control information from the control information output unit 13 tothe feedback device 3.

By the feedback control information, the feedback reproduction controlunit 3 b of the feedback device 3 controls the driving of the tactileimpetus reproduction with respect to the respective reproduction devices4 mounted on the respective users.

As the user's motions are scattered (or coincide with each other on thecontrary), the feedback reproduction strength is heightened.

By this, the respective users is able to recognize the state of thecorrelation of all the movements through the strength of the tactileimpetus reproduction.

The operation unit 11, after the processing in step F206, returns fromstep F207 to step F202 to continue the same process.

If the system operation is ended, the operation unit 11 proceeds fromstep F207 to step F208, and ends the recording process of the movementinformation in the recording unit 12, the operation process of theoperation unit 11, and the generation/output process of the feedbackcontrol information to terminate a series of operations.

In the second processing example, the whole discrepancy is determinedfrom the discrepancies between the user's combinations in any place, andthe feedback reproduction according to the whole discrepancy isperformed.

Of course, the operation may be performed according to the correlationrather than the discrepancy. Also, the discrepancy or the correlationmay be determined from the movement pattern, the rhythm pattern, and themovement strength rather than the movement tempo.

4. Third Processing Example

The third processing example of the motion coordination operation device1 will be described with reference to FIG. 10.

Steps F301, F302, and F303 in FIG. 10 are the same as steps F101, F102,and F103 of the processing example 1, and thus the explanation thereofwill be omitted.

In step F304, the operation unit 11 estimates the movement rhythm of aspecified user. For example, if one of the plurality of users is aleader in dances or the like, the user is determined as the specifieduser. The operation unit 11 estimates the movement rhythm from themovement information from the sensor devices 2 mounted on the specifieduser.

At this time, in practice, in step F303, the movement rhythm (movementtempo) is determined as the individual information of all the users, andin step F304, the movement rhythm of the specified user is selectedamong the movement rhythms of all the users.

In step F305, the operation unit 11 calculates the discrepancy betweenthe movement rhythm of the specified user and the movement rhythm ofeach user. For example, if the user A is the specified user, thediscrepancies in the movement rhythm between the specified user A andanother user, such as between the users A and B, between the users A andC, between the users A and D, and the like, are calculated.

For example, if it is assumed that with respect to the movement temposbetween the specified user and another user, “tempoα” is the side havinga fast tempo, and “tempol3” is the side having a slow tempo, thediscrepancy is obtained as follows.

Discrepancy=tempoα/tempoβ−int(tempoα/tempoβ)

If the discrepancies of the tempos of the respective other users withrespect to the tempo of the specified user A become greater, thediscrepancy values become larger.

Also, in step F306, the operation unit 11 generates the feedback controlinformation, and transmits the feedback control information from thecontrol information output unit 13 to the feedback device 3.

The feedback control information is generated according to thediscrepancies between the specified user and other users except for thespecified user. That is, the tactile impetus strengths are set for therespective users according to the discrepancy values.

The operation unit 11, as the feedback control information, generatesthe tempo information as the movement rhythm of the specified user A andthe strength coefficient according to the discrepancy values of therespective users, and outputs the control information to the feedbackdevice 3.

The strength coefficient may be the discrepancy value itself, a valueobtained by multiplying the discrepancy value by a predeterminedcoefficient, or a value obtained by multiplying them by a time value.

By the feedback control information, the feedback reproduction controlunit 3 b of the feedback device 3 controls the driving of the tactileimpetus reproduction with respect to the respective reproduction devices4 mounted on the respective users.

That is, although the tempo as the movement rhythm of the specified userA is transferred to the tactile sensation with respect to the usersexcept for the specified user A, the tactile impetus becomes greater asthe deviation of the motions from the specified user A becomes larger.That is, the user who has larger deviation of the motions from thespecified user A feels the tempo of the specified user A stronger.

At this time, although it is reasonable to do so for this, it isnecessary that the control information for the respective userscorrespond to the reproduction devices 4 mounted on the respectiveusers. The feedback reproduction control unit 3 b sets the reproductionstrength of the reproduction devices 4 mounted on the user, and controlsthe driving of the reproduction device 4 according to the controlinformation (strength coefficient according to the discrepancy) for therespective users.

If the tempos for the respective users coincide with that of thespecified user A (for example, a dance leader or the like) as a resultof comparison, the user feels small with respect to the rhythm felt withtactile sensation by the reproduction device 4, while if the temposdeviate from that of the specified user A, the user feels large withrespect to the rhythm felt with tactile sensation.

The operation unit 11, after the processing in step F306, returns fromstep F307 to step F302 to continue the same process.

If the system operation is ended, the operation unit 11 proceeds fromstep F307 to step F308, and ends the recording process of the movementinformation in the recording unit 12, the operation process of theoperation unit 11, and the generation/output process of the feedbackcontrol information to terminate a series of operations.

In the third processing example as described above, if the basicmovement rhythm felt with tactile sensation becomes strong, the userfeels that the user's motion deviates from the motion of a specifieduser, for example, the leader, and performs the dance with attention tothe consistency to the leader. Accordingly, the user can be aware of thecoincidence state with the leader as performing the operation, andquickly learn the movement. Also, the user can take pleasure inincreasing the correlation of the motion.

In this case, the operation process may be performed by correlationrather than the discrepancy. Also, the discrepancy or the correlationmay be determined from the movement pattern, the rhythm pattern, and themovement strength rather than the movement tempo.

Also, the feedback reproduction is not performed as the tactile impetusreproduction with the strength individually different for each user, butmay be commonly reproduced to all the users.

For example, in steps F305 and F306, the strength of the discrepancy isset according to the width of distribution of movement rhythms of therespective users with respect to the movement rhythm of the specifieduser A or according to the average value of the discrepancies of therespective users.

That is, the strength coefficient of the feedback control information isheightened as the motions of the respective users get scattered.

Accordingly, all the users strongly feel the tactile impetusreproduction as the deviation from the leader becomes greater, andbodily sense the coincident state of the motions of all the userssimultaneously moving, imitating the leader.

5. Fourth Processing Example

The fourth processing example will be described with reference to FIG.11.

In the fourth processing example, the feedback apparatus 3 gives theguidance rhythm to the users as the tactile impetus under the control ofthe guidance device 5.

In step F401 as the processing of the motion coordination operationdevice 1, the motion coordination operation device 1 starts therecording of the movement information. That is, if the motioninformation input unit 14 of the motion coordination operation device 1receives the movement information from the respective sensor devices 2,the motion coordination operation device 1 records the received motioninformation in the recording unit 12. Thereafter, such a recordingprocess is continuously performed.

In step F402, the operation unit 11 generates control information forthe impetus reproduction of the guidance tempo, and outputs the controlinformation from the control information output unit 13 to the guidancedevice 5 and the feedback device 3. That is, the operation unit 11instructs the necessary guidance tempo to the guidance device 5. Also,the operation unit 11 instructs starting of the tactile impetusreproduction of the guidance temp to the feedback apparatus 3. Theguidance control unit 5 a instructs the guidance tempo to the feedbackreproduction control unit 3 b according to the instruction.

Accordingly, the tactile impetus of the predetermined guidance tempo isexecuted by the respective reproduction devices 4, and thus therespective users bodily sense the guidance tempo.

At this time, it is also possible for the guidance reproduction unit 5 bsimultaneously performs the reproduction according to the guidance tempoby audio or video.

In step F403, the operation unit 11 reads the movement information for apredetermined period with respect to the movement information from therespective sensor devices 2, which is recorded in the recording unit 12.

In step F404, the operation unit 11 performs the individual informationoperation of the users of the respective sensor devices 2 using themovement information from the respective sensor devices 2, which is readfrom the recording unit 12. For example, the operation unit 11 performsthe estimation of the movement rhythms. The steps F403 and F404 asdescribed above are the same as steps F102 and F103 according to theprocessing example 1.

Next, in step F405, the operation unit 11 calculates the discrepancybetween the tempo as the movement rhythm of the respective users and theguidance tempo.

For example, if it is assumed that with respect to the guidance temposand the movement tempos of the respective users, “tempoα” is the sidehaving a fast tempo, and “tempoβ” is the side having a slow tempo, thediscrepancy is obtained as follows.

Discrepancy=tempoα/tempoβ−int(tempoα/tempoβ)

If the discrepancy between the user's tempo and the guidance tempobecomes greater, the discrepancy value becomes a larger value.

If the discrepancies for the respective guidance tempos are obtainedwith respect to the users, in step F406, the operation unit 11calculates the discrepancy in the whole group. For example, theoperation unit 11 considers the average value of the discrepancies ofthe respective users or the distribution width as the discrepancy of thewhole group.

In step F407, the operation unit 11 generates the feedback controlinformation, and transmits the feedback control information from thecontrol information output unit 13 to the feedback device 3.

The feedback control information is generated according to thediscrepancy of the whole group.

For example, the operation unit 11 generates the feedback controlinformation having high reproduction strength coefficient as thediscrepancy of the whole group becomes larger (or smaller), and outputsthe control information from the control information output unit 13 tothe feedback device 3.

By the feedback control information as described above, the feedbackreproduction control unit 3 b of the feedback device 3 controls thedriving of the tactile impetus with respect to the respectivereproduction devices 4 mounted on the respective users.

Then, as the user's motions are scattered (or coincide with each otheron the contrary) with respect to the guidance tempo, the feedbackreproduction strength is heightened.

By this, the respective users are able to recognize the state of thecorrelation of the movements through the strength of the tactile impetusreproduction.

The operation unit 11, after the processing in step F407, returns fromstep F408 to step F403 to continue the same process.

If the system operation is ended, the operation unit 11 proceeds fromstep F408 to step F409, and ends the recording process of the movementinformation in the recording unit 12, the operation process of theoperation unit 11, and the generation/output process of the feedbackcontrol information to terminate a series of operations.

In the fourth processing example, the whole discrepancy of the motionsof all the users is determined as the guidance tempo is recognized bythe user, and the feedback reproduction according to the wholediscrepancy is performed.

Of course, the operation may be performed according to the correlationrather than the discrepancy. Also, the discrepancy or the correlationmay be determined from the movement pattern, the rhythm pattern, and themovement strength rather than the movement tempo.

6. Applications, Modifications, and Program

As described above, various kinds of processing examples have beendescribed as embodiments of the present invention. Here, applicationsand modifications in various kinds of the processing examples will bedescribed.

As the applications of the embodiment of the present invention,gymnastics or dances which are performed as a group as described abovemay be considered. In the case where a plurality of persons performdancing, in the related art, a supervisor observes whether the motionsof the dancing persons coincide with one another or the movements arefilmed on video to be confirmed later. By using the system according tothe examples of the present invention, the dance rhythms may betransferred to a performer in real time or in advance, and whether theperformed motions are in harmony with those of other persons can be fedback.

In addition, synchronized swimming, marching, person's motions in a liveplace, and a rhythm game may be considered.

It is sufficient if the feedback reproduction is generated based on thesynchronization of the movement state of the plural users.

It is also considered whether user movement tempos, movement strengths,movement regions, and movement patterns coincide with each other in apredetermined range, or in the case of the tempo, if the rhythm iscomposed of up beat and harmonic, whether to give predetermined feedbackto the users.

In the tactile impetus reproduction, the strength and weakness of a beatfelt by a pressure device, beat intervals, vibration strength, vibrationtime, and the like, are changed by the synchronization state.

Also, in the case of performing the feedback reproduction with sound ormusic, for example, if the movement patterns coincide with each other,it is possible to output noisy sound by overlapping a plurality ofsounds in the next measure, or to give announcement to the effect thatthe movements harmonize with each other.

If the movement pattern is early or late, an announcement to the effectthat the overlapping of music does not occur in the next measure,dissonance occurs, or the movements do not harmonize with each other,may be given. In the case were the rhythm is composed of up beat, it isalso possible to add a beat sound or rhythm sound.

As the correlation becomes greater, the number of overlapping chords orvolume becomes larger. On the other hand, as the correlation becomessmaller, the number of overlapping chords or volume also becomessmaller, and the number of dissonances or the volume becomes larger.

The sensor device 2 may be mounted on or carried by the user. However,for example, it is also possible to image the user's motion by a cameradevice and to determine the motion of the respective users by the imagerecognition process.

Also, for example, in the processing example 4, it is also consideredthat as the guidance rhythm or the reproduction by the guidance device 5that is independent from the system operation, the next rhythm ormovement direction are enlightened by giving the tactile impetus when atemplate exists already.

Also, when the harmonized movement has succeeded, information that hasnot been performed is recorded, and thus the user's motion may be turnedabout.

The motion coordination operation device 1 records the sensor outputvalues in the recording unit 12 together with the time information.Based on this log, the time series movement pattern and feedback may bereproduced.

It is also possible to look back on the correlation between the user andanother user or the discrepancy in motion between the user and thepre-registered template, and to use the log as the template.

The user may prepare the template of the movement.

By the manipulation of the sensor device 2 (for example, accelerationsensor) attached to the body or by starting a dedicated software, thetemplate reproduction manipulation can be performed on the software, andthus the movement thereafter may be recorded.

Also, although it is exemplified that the sensor devices 2 are mountedon the users as shown in FIG. 2 in the processing examples 1, 2, 3, and4, the plurality of sensor devices 2 may be mounted on each user in thesame manner as shown in FIGS. 3 and 4.

For example, in the processing example 1 of FIG. 5, the operations MIAand M1B in FIG. 3 are executed in step F103.

Also, in the processing example 2 of FIG. 9, the operations M2A and M2Bof FIG. 4 are executed in step F204.

In other processing examples, by properly performing the operations ofFIGS. 3, and 4, it is possible for the user to mount the plurality ofsensor devices 2 thereon.

Also, in the embodiments, it is exemplified that the motion coordinationoperation device 1 records the movement information in the recordingunit 12. However, according to the processing capability of theoperation unit 11, the operation unit 11 may directly process themovement information received in the movement information input unit 14without taking the recording operation.

The program will be described.

In the embodiment of the present invention, the program is a programthat realizes the function of the above-described motion coordinationoperation device 1. For example, the program, which is installed in thecomputer device, starts to make the computer device execute theprocesses as in the processing examples 1, 2, 3, and 4.

In this embodiment, the program may be pre-recorded in an HDD as arecording medium built in an appliance such as a personal computer, adedicated motion coordination operation device 1, or a ROM in amicrocomputer having a CPU.

Also, the program may be temporarily or permanently stored in a flexibledisc, CD-ROM (Compact Disc Read Only Memory), MO (Magnet Optical) disc,a DVD (Digital Versatile Disc), a Blu-ray disc, a magnetic disc, asemiconductor memory, a removable recording medium such as a memorycard. The removable recording medium may be provided as so-calledpackage software.

Also, the program may be downloaded from a download site through anetwork such as LAN (Local Area Network), Internet, or the like, inaddition to the installation in the personal computer from the removablerecording medium.

According to the corresponding program, the motion coordinationoperation device 1 may be realized using an information processingdevice such as a personal computer, and is suitable to the wide-rangedproviding of the system operation.

The present application contains subject matter coordination to thatdisclosed in Japanese Priority Patent Application JP 2009-244304 filedin the Japan Patent Office on Oct. 23, 2009, the entire content of whichis hereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An information processing apparatus comprising: circuitry configuredto acquire movement information of a plurality of users; determine afirst information indicating the synchronization of motions of bodies orparts of the bodies of the plurality of users based on the movementinformation; and determine feedback outputs based on the firstinformation; wherein the feedback outputs are communicated to theplurality of users via tactile sensation.
 2. The information processingapparatus according to claim 1, wherein the circuitry is configured toacquire the movement information from respective sensor devices mountedon the plurality of users.
 3. The information processing apparatusaccording to claim 2, wherein the sensor device is mounted on a wrist ofuser, an ankle of user, a neck of user, an arm of user, a trunk of useror a leg portion of user.
 4. The information processing apparatusaccording to claim 1, wherein the circuitry is configured to acquire themovement information from respective sensor devices, a type of thesensor device is one of headphone type, a necklet type that is put on aneck, a type that is mounted on clothes with a clip or a type that isput in a pocket.
 5. The information processing apparatus according toclaim 1, wherein the circuitry is configured to acquire the movementinformation from respective sensor devices held by user in user's hand.6. The information processing apparatus according to claim 1, whereinthe movement information are based on images captured by camera devices.7. The information processing apparatus according to claim 1, whereinthe synchronization is measured by comparing the motions of theplurality of users to one another.
 8. The information processingapparatus according to claim 1, wherein respective degrees of thefeedback outputs are determined based on the first information.
 9. Aninformation processing apparatus comprising: circuitry configured toacquire movement information of a plurality of users; determine a firstinformation indicating the synchronization of motions of bodies or partsof the bodies of the plurality of users based on the movementinformation; and determine feedback outputs based on the firstinformation; wherein respective degrees of the feedback outputs aredetermined based on the first information.
 10. The informationprocessing apparatus according to claim 9, wherein the circuitry isconfigured to acquire the movement information from respective sensordevices mounted on the plurality of users.
 11. The informationprocessing apparatus according to claim 10, wherein the sensor device ismounted on a wrist of user, an ankle of user, a neck of user, an arm ofuser, a trunk of user or a leg portion of user.
 12. The informationprocessing apparatus according to claim 9, wherein the circuitry isconfigured to acquire the movement information from respective sensordevices, a type of the sensor device is one of headphone type, a necklettype that is put on a neck, a type that is mounted on clothes with aclip or a type that is put in a pocket.
 13. The information processingapparatus according to claim 9, wherein the circuitry is configured toacquire the movement information from respective sensor devices held byuser in user's hand.
 14. The information processing apparatus accordingto claim 9, wherein the movement information are based on imagescaptured by camera devices.
 15. The information processing apparatusaccording to claim 9, wherein the synchronization is measured bycomparing the motions of the plurality of users to one another.
 16. Theinformation processing apparatus according to claim 9, wherein thefeedback outputs are communicated with the plurality of users via one oftactile sensation, audio or image.
 17. A method comprising: acquiringmovement information of a plurality of users; determining a firstinformation indicating the synchronization of motions of bodies or partsof the bodies of the plurality of users based on the movementinformation; and determining feedback outputs based on the firstinformation; wherein the feedback outputs are communicated to theplurality of users via tactile sensation.